A Proposal Concerning the Analysis of Shadows in Images by an Active Observer (Dissertation Proposal)

Shadows occur frequently in indoor scenes and outdoors on sunny days. Despite the information inherent in shadows about a scene\u27s geometry and lighting conditions, relatively little work in image understanding has addressed the important problem of recognizing shadows. This is an even more serious failing when one considers the problems shadows pose for many visual techniques such as object recognition and shape from shading. Shadows are difficult to identify because they cannot be infallibly recognized until a scene\u27s geometry and lighting are known. However, there are a number of cues which together strongly suggest the identification of a shadow. We present a list of these cues and methods which can be used by an active observer to detect shadows. By an active observer, we mean an observer that is not only mobile, but can extend a probe into its environment. The proposed approach should allow the extraction of shadows in real time. Furthermore, the identification of a shadow should improve with observing time. In order to be able to identify shadows without or prior to obtaining information about the arrangement of objects or information about the spectral properties of materials in the scene, we provide the observer with a probe with which to cast its own shadows. Any visible shadows cast by the probe can be easily identified because they will be new to the scene. These actively obtained shadows allow the observer to experimentally determine the number and location of light sources in the scene, to locate the cast shadows, and to gain information about the likely spectral changes due to shadows. We present a novel method for locating a light source and the surface on which a shadow is cast. It takes into account errors in imaging and image processing and, furthermore, it takes special advantage of the benefits of an active observer. The information gained from the probe is of particular importance in effectively using the various shadow cues. In the course of identifying shadows, we also present a new modification on an image segmentation algorithm. Our modification provides a general description of color images in terms of regions that is particularly amenable to the analysis of shadows

Active Color Image Analysis for Recognizing Shadows

Many existing computer vision modules assume that shadows in an image have been accounted for prior to their application. In spite of this, relatively little work has been done on recognizing shadows or on recognizing a single surface material when directly lit and in shadow. This is in part because shadows cannot be infallible recognized until a scene\u27s lighting and geometry are known. However, color is a strong cue to the presence of shadows. We present a general color image segmentation algorithm whose output is amenable to the recovery of shadows as determined by an analysis of the physics of shadow radiance. Then, we show how an observer that can cast its own shadows can infer enough information about a scene\u27s illumination to refine the segmentation results to determine where the shadows in the scene are with reasonable confidence. Having an observer that can actively cast shadows frees us from restrictive assumptions about the scene illumination or the reliance on high level scene knowledge. We present results of our methods on images of complex indoor and outdoor scenes

Final Report on Advanced Research in Range Image Interpretation for Automated Mail Handling

We discuss an implemented software system that interprets dense range images obtained from scenes of heaps of postal pieces: letter, parcels, etc. We describe a model-based system consisting of segmentation, modeling, and classification procedures. First, the range image is segmented into regions and reasoning is done about the physical support of these regions. Second, for each region several possible 3-D interpretations are made based on the partial knowledge which is updated whenever a new interpretation is obtained. Finally each interpretation is tested against the data for its consistency. We have chosen the superquadric model as our 3-D shape descriptor, plus deformations such as tapering and bending along the major axis. The superquadric model is an analytic representation of volume for which a cross-section is one of a class of curves varying between rectangular to elliptical shaped. Superquadric parameters are recovered by minimizing the least-squares error between the superquadric surface and the range data. The system recovers p position, orientation, shape, size and class of the object. Using the goodness of fit and Euclidean distance measures, and the shape and size parameters of the recovered model, objects are classified into one of the following broad categories: flat (letters), box (parcels), roll (circular and elliptical cylinders), and irregular (film mailers etc.). The overall approach to this problem has been to find the most general yet computationally economical method to interpret the data. Experimental results obtained from a large number of complex range images are reported

The Visual Recognition Of Shadows By An Active Observer

In computer vision for object recognition or autonomous navigation, shadows are a frequent occurrence. However, shadows in an image can make it difficult to partition the image into regions corresponding to physical objects. Consequently, shadows must be accounted for in images. Despite this, relatively little work in image understanding has addressed the problem of recognizing shadows. This is in large part because shadows are difficult to identify. They cannot be infallibly recognized until a scene's geometry and lighting are known. However, this dissertation present a number of cues which together strongly suggest the identification of a shadow and which can be examined without a high computational cost. The techniques developed are: a color model for shadows and a color image segmentation method that recovers single material surfaces as single image regions irregardless of whether the surface..

Combining Color and Geometry for the Active, Visual Recognition of Shadows

In computer vision for object recognition or navigation, shadows are a frequent occurrence. However, shadows are difficult to recognize because they cannot be infallibly recognized until a scene's geometry and lighting are known. We present a number of cues which together strongly suggest the identification of a shadow and which can be examined without a high computational cost. The techniques developed are: a color model for shadows and a color image segmentation method that recovers single material surfaces as single image regions irregardless of whether the surface is partially in shadow; a method to recover the penumbra and umbra of shadows; and, a method for determining whether some object could be obstructing a light source. These cues either depend on or their reliability improves with the examination of some well understood shadows in a scene. Our observer is equipped with an extendable probe for casting its own shadows. These actively obtained shadows allow the observer to exp..

Graph Cuts and Efficient N-D Image Segmentation

Combinatorial graph cut algorithms have been successfully applied to a wide range of problems in vision and graphics. This paper focusses on possibly the simplest application of graph-cuts: segmentation of objects in image data. Despite its simplicity, this application epitomizes the best features of combinatorial graph cuts methods in vision: global optima, practical efficiency, numerical robustness, ability to fuse a wide range of visual cues and constraints, unrestricted topological properties of segments, and applicability to N-D problems. Graph cuts based approaches to object extraction have also been shown to have interesting connections with earlier segmentation methods such as snakes, geodesic active contours, and level-sets. The segmentation energies optimized by graph cuts combine boundary regularization with region-based properties in the same fashion as Mumford-Shah style functionals. We present motivation and detailed technical description of the basic combinatorial optimization framework for image segmentation via s/t graph cuts. After the general concept of using binary graph cut algorithms for object segmentation was first proposed and tested in Boykov and Jolly (2001), this idea was widely studied in computer vision and graphics communities. We provide links to a large number of known extensions based on iterative parameter re-estimation and learning, multi-scale or hierarchical approaches, narrow bands, and other techniques for demanding photo, video, and medical applications

An energy minimization approach to the data driven editing of presegmented images/volumes

Fully automatic, completely reliable segmentation in medical images is an unrealistic expectation with today’s technology. However, many automatic segmentation algorithms may achieve a near-correct solution, incorrect only in a small region. For these situations, an interactive editing tool is required, ideally in 3D, that is usually left to a manual correction. We formulate the editing task as an energy minimization problem that may be solved with a modified version of either graph cuts or the random walker 3D segmentation algorithms. Both algorithms employ a seeded user interface, that may be used in this scenario for a user to seed erroneous voxels as belonging to the foreground or the background. In our formulation, it is unnecessary for the user to specify both foreground and background seeds